Un-defer fan-out per-instance fixture 029#188
Merged
Conversation
029's fan_out omits the collect_field/target_field/item_field the cross-cap adapter requires, and its inner subgraph ships no state block. Synthesize those into a throwaway aggregation sink (029 asserts per-instance span metadata + sibling isolation, never the collected results) and seed the inner state, then activate 029 in the Langfuse conformance runner. Rewrite _make_augment_instance_middleware to read each instance's item from its item_field slot in runtime state. The prior approach indexed a build-time list by current_fan_out_index(), which returns None inside instance middleware (the ContextVar is set deeper, in node execution), so the augmentation never fired -- latent because 029/039 were its only users and both were deferred. Sharpen the 030/039 deferral notes: 030 needs a LangfuseObserver per-branch dispatch-span src change, 039 needs a runtime-state item lookup plus a new augment_metadata_from_outer_item directive.
There was a problem hiding this comment.
Pull request overview
This PR activates observability conformance fixture 029 for the Langfuse runner by extending the Langfuse harness to (a) normalize fixture keys, (b) synthesize adapter-required fan-out aggregation/state fields, and (c) fix per-instance metadata augmentation so it runs in fan-out instance middleware.
Changes:
- Moved fixture 029 into the Langfuse harness coverage set and removed it from the unit-tested/deferred list.
- Added
_synthesize_fan_out_aggregationto inject required fan-out fields + inner state seed for fixture 029. - Updated fan-out per-instance augmentation middleware to read the current item from runtime state (
item_field) instead of a build-time list indexed bycurrent_fan_out_index().
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/conformance/test_observability.py | Marks fixture 029 as covered by the sibling Langfuse conformance runner rather than deferred/unit-only. |
| tests/conformance/test_observability_langfuse.py | Activates fixture 029 and updates the Langfuse harness to synthesize missing fan-out/state requirements and fix per-instance augmentation middleware. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
From CoPilot review of #188: - Make the inner-state synthesis additive: ensure item_field and the inferred response fields are declared whether or not the subgraph shipped its own state block. setdefault on the whole state block skipped a fixture that declares inner state without the item slot, which would fail strict-State validation when the engine writes it. - Fail fast when augment_metadata_from_field is present but item_field is missing, rather than building a silent no-op middleware that hides a harness/fixture mismatch. Tighten the middleware signature to a non-optional item_field.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Un-defers fixture 029 (caller-metadata fan-out per-instance) in the Langfuse conformance runner.
Why it was deferred
029 omits the
collect_field/target_field/item_fieldthe cross-cap adapter requires, and its inner subgraph ships nostate:block. Beyond that, the augment-instance-middleware the harness uses foraugment_metadata_from_fieldwas non-functional: it indexed a build-time list bycurrent_fan_out_index(), which returnsNoneinside fan-out instance middleware (the ContextVar is set deeper, in node execution). That path was latent dead code, since 029 and 039 were its only users and both were deferred.Changes
_synthesize_fan_out_aggregation: synthesize the adapter-required fan-out fields into a throwaway aggregation sink (029 asserts per-instance span metadata + sibling isolation, never the collected results) and seed the inner state._make_augment_instance_middleware: read each instance's item from itsitem_fieldslot in runtime state, so the augmentation fires before the inner spans open and lands on the instance's dispatch + inner spans._LANGFUSE_FIXTURES; move it to_LANGFUSE_HARNESS_FIXTURESintest_observability.py.augment_metadata_from_outer_itemdirective). Both are follow-up PRs.Test plan
uv run pytest "tests/conformance/test_observability_langfuse.py" -k 029-> 1 passed.tests/-> 1460 passed. ruff + pyright clean.Test-only; no behavior or pin change.